Feat 698 equity points - #714
Merged
Merged
Conversation
#698) `agent._mark_to_market_equity` has always run once per cycle, and nothing kept the answer. What survived was `agent_state["equity_history"]`: a 7-day window that exists for the weekly drawdown rail and that `record_external_flow` REWRITES on a declared deposit. That is a rail's working set, not a record -- so the series is written alongside it rather than derived from it, and neither reconstructs the other. `equity_points` (schema v19) is the record. Per row: the reading, the mode that produced it, and the high-water mark in force AFTER it was folded in -- read back by the chart rather than recomputed, because a rebased HWM is not the running maximum of the equity and the overlay's whole point is the ceiling rail 11 actually had. `mode` is the load-bearing partition, taken from the same `equity_state_mode` stamp `_clear_live_mode_if_needed` reads before wiping the shared HWM on a flip. Deriving it a second way at the call site would let two answers to one question drift, and the failure is silent: a mislabelled row does not go missing, it lands in the other account's curve. An UNSTAMPED mode writes nothing and never guesses -- and never raises, so the chart can never take rail 11 down with it. `cash` and `unrealized` are the split, and they are passed, never derived: this function has a total and no positions. NULL means "not recorded", never zero -- the `orders.filled_quantity` convention. Both agent branches now supply them from ONE pass over ONE set of balances and marks (`_mark_to_market_parts`, `PaperTrader.unrealized`), so a price cannot tick between the total and its parts and file a row that does not reconcile. `unrealized_on_marks` mirrors `mark_positions` guard for guard: a position valued at cost contributes zero, because the equity in the same row valued it at cost too. Tested as an identity: cash + cost basis + unrealized == equity. NO BACKFILL, and what could be backfilled is exactly what must not be. Replaying the shifted 7-day window would publish equities the account never had; the orders ledger would give a closed-trade curve with no cash leg, a different quantity wearing this table's name. An empty table means "not observed before v19", which is true. `build_equity_series` is a NEW builder, not a change to `build_equity_curve`. That one plots cumulative net P&L over closed trades and argues for its trade-order axis on its own terms -- correctly, for that quantity. This plots what the account was worth whether or not it traded, where the gaps between cycles are the information. Segmented by RUNS of mode (paper, live, paper is three), bounds spanning the overlays so a rail floor cannot fall off the box, and zero deliberately NOT forced onto the axis: an account is not up or down against nothing. Three departures from the issue text: - v19, not v16: main reached 18 after the issue was filed. - `ts INTEGER`, not TEXT: every timestamp in this schema is an epoch int, and TEXT orders epochs lexically -- the `ts >= ?` window the chart reads by would silently return the wrong rows across a digit-count change. Pinned by a test. - the record is `EquityReading`: `keel.commands.insights` already owns an `EquityPoint`, a PLOT point, and the chart module now uses both. Still to come on #698: the payload/template/chart.js wiring that puts the series on the Insights page. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…curve (#698) Portfolio reality above statistical expectancy. `equityChart` keeps plotting cumulative net P&L over closed TRADES on a trade-order axis -- `build_equity_curve` argues for that axis on its own terms, and the argument is still right for that quantity. Above it now sits what the ACCOUNT was worth every cycle, traded or not, on a time axis, where the quiet week is the information rather than noise. Two questions, two charts; neither replaces the other. payload.py -- `equity_series_payload`, and `insights_payload` gains a REQUIRED `series=` keyword for the reason `journal_payload`'s `curve=` is one: a default would quietly serve a view with no chart. Segments cross AS SEGMENTS, never as one flat list; the mode partition is the whole reason the shape exists, and flattening is the single operation this wire must not make easy. Coordinates are bare strings, everything a reader is told is a `Field`, and `dd_floor_y` is `null` rather than `"0"` when the rail setting is unknown -- zero is the TOP of an SVG box, so a `"0"` would draw a ceiling in force above every reading. The reading -- the sentence a reader who cannot see the chart is given -- names the modes, because that is the part a spoken summary would otherwise flatten into "an account went from ten thousand dollars to two hundred and fifty". It says the lines are separate accounts ONLY when there is a partition to explain: told to a paper-only deployment it sends a reader looking for a line that is not there. `state` is neutral, unlike the curve's: a curve closes on a P&L, which is a verdict, and a series closes on a balance, which is not one. `modes` and `is_partitioned` are properties of `EquitySeries`, not a list the serialiser assembles. Rule 6e bans `len()` in payload.py so that a count on the wire is one the report already holds, and the first draft of that sentence broke it -- the rule was written down in `EquityCurve.point_count`'s own docstring the whole time. api.py -- the series is read on `/api/insights`, not on `/api/journal` beside the curve it is drawn above, because it describes the ACCOUNT and not the closed trades. The journal's curve narrows with that endpoint's `?limit=`; this narrows with nothing, and two charts that answer a query differently must not share a payload where a reader would assume they agree. `max_total_dd_pct` comes off the same loaded config `build_insights_report` reads, so the floor under the chart and the ceiling quoted in the account card beside it are one setting rather than two reads that can drift. Every recorded cycle, never a default window: cutting it to a span here would reintroduce the exact horizon this issue exists to remove. chart.js -- `equitySeriesChart`: one polyline per segment, plus the high-water mark and the drawdown floor drawn beneath the account line in the token the baseline already uses. The canvas is `svg.series` and NOT `svg.curve` on purpose: #602's wheel-zoom, drag-to-pan and cursor legend reach for `svg.curve` with `querySelector`, which takes the first match, and this chart renders first. Paper is dashed and live is solid, so the two accounts stay distinguishable in greyscale, on e-ink and with red-green colour deficiency -- the same rule the losing-segment dash already follows, and the same principle as payload.py's ▲/▼. render.js -- stacked, with a note saying the top chart covers every recorded cycle rather than the journal's row cap, since two charts side by side otherwise invite the assumption that the cap applies to both. A payload with no `equity_series` at all draws nothing and says nothing: that is not a deployment with no readings, it is a response from before the field existed, which a service worker can still be holding after an upgrade -- and reading `.reading.display` off it would blank the whole view over a stale cache entry that the next refresh fixes on its own. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…able unbounded (#698) Two review findings on the equity series, plus the test that was missing. THE REGRESSION. `equitySeriesChart` set `figure.className = "chart"`, and `main.js` reaches for the chart's WRAPPER twice -- `highlightJournalRow` and the chart-action handler -- with `contentNode.querySelector("figure.chart")`, which takes the first match in document order. The series is appended ABOVE the curve, so both lookups landed on it; `highlightTrade` then found no `.highlight` group, returned early, and hovering a journal row on /insights silently highlighted nothing. It worked before this branch, so it was a pure regression. The original reasoning got the CANVAS right and stopped there: `svg.series` was chosen deliberately so the #602 pointer gestures could not retarget, and the comment says so at length. It never asked what else `main.js` selects on. The figure is now `"chart series"` and both lookups carry `:not(.series)`; a test pins both halves, because either alone leaves the collision live. THE UNBOUNDED READ. `/api/insights` read the whole `equity_points` table on every request. The table is append-only and grows one row per cycle forever -- at the default `auto_trade.interval_sec` of 900 that is ~35,000 rows a year -- and `main.js` re-polls the view every 15 seconds, rebuilding one SVG circle per point each time. Measured at 580 bytes per point in the rendered payload, a year of the hourly paper profile is 4.8 MiB per response, re-fetched four times a minute. The previous note called the unbounded read deliberate ("every recorded cycle, never a default window"); the choice was defensible and its cost was not bounded, which is the half that did not hold. `EQUITY_POINT_LIMIT = 1000`, newest first, sized off `PLOT_WIDTH`: past one reading per plot unit the extra rows land on coordinates already drawn, so they cost bytes with nothing on screen to show for it. A fixed cap rather than a `?limit=` because this route carries no collection to page and the whole series is on disk for anyone who wants it. A BOUNDED READ HAS TO SAY SO. Silently starting the line wherever the cap fell would misstate the span of the record while every individual point stayed true -- the harder kind of wrong to notice, and the kind this chart exists to avoid. So `count_equity_points()` rides along, `EquitySeries` carries `total_recorded` and `is_truncated`, the spoken reading says "the most recent N" and names the total, and the note under the chart reads "N of M recorded cycle(s)". `total_recorded=None` means the caller did not say, which is deliberately NOT a claim of completeness: inventing "complete" from silence is the failure that guard exists for. `get_equity_points(limit=)` keeps the most recent N and still returns them oldest first -- the ordering is the caller's contract, not an artefact of how rows were selected -- and the limit applies inside the mode filter, so asking for the last two live readings on a paper-heavy database cannot return nothing. THE MISSING TEST. All three agent tests were on the live branch. The paper branch now has one through the real loop, asserting the recorded parts add back to the recorded total on a cycle whose account seeded and traded before the reading -- the case where a total and a split could come from two different states of the account. Both halves of the wiring were mutation-checked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Tests-first evidence
Gates (all must pass)
uv run ruff checkcleanuv run mypycleanuv run pytest -qgreenScope check
leave checked only if true, and if so: cite the source and open the discussion
BEFORE review (CONTRIBUTING.md, "Governance: rulings vs. machinery").